Teaching Robots Novel Objects by Pointing at Them

Robots that must operate in novel environments and collaborate with humans must be capable of acquiring new knowledge from human experts during operation. We propose teaching a robot novel objects it has not encountered before by pointing a hand at the new object of interest. An end-to-end neural network is used to attend to the novel object of interest indicated by the pointing hand and then to localize the object in new scenes. In order to attend to the novel object indicated by the pointing hand, we propose a spatial attention modulation mechanism that learns to focus on the highlighted object while ignoring the other objects in the scene. We show that a robot arm can manipulate novel objects that are highlighted by pointing a hand at them. We also evaluate the performance of the proposed architecture on a synthetic dataset constructed using emojis and on a real-world dataset of common objects

Open Arms: Open-Source Arms, Hands & Control

Open Arms is a novel open-source platform of realistic human-like robotic hands and arms hardware with 28 Degree-of-Freedom (DoF), designed to extend the capabilities and accessibility of humanoid robotic grasping and manipulation. The Open Arms framework includes an open SDK and development environment, simulation tools, and application development tools to build and operate Open Arms. This paper describes these hands controls, sensing, mechanisms, aesthetic design, and manufacturing and their real-world applications with a teleoperated nursing robot. From 2015 to 2022, the authors have designed and established the manufacturing of Open Arms as a low-cost, high functionality robotic arms hardware and software framework to serve both humanoid robot applications and the urgent demand for low-cost prosthetics, as part of the Hanson Robotics Sophia Robot platform. Using the techniques of consumer product manufacturing, we set out to define modular, low-cost techniques for approximating the dexterity and sensitivity of human hands. To demonstrate the dexterity and control of our hands, we present a Generative Grasping Residual CNN (GGR-CNN) model that can generate robust antipodal grasps from input images of various objects in real-time speeds (22ms). We achieved state-of-the-art accuracy of 92.4% using our model architecture on a standard Cornell Grasping Dataset, which contains a diverse set of household objects.Comment: Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022

An animation-and-chirplet based approach to intruder classification using PIR sensing

The development of a Passive Infra-Red (PIR) sensing based intrusion detection system is presented here having the ability to reject vegetative clutter and distinguish between human and animal intrusions. This has potential application to reducing human-animal conflicts in the vicinity of a wildlife park. The system takes on the form of a sensor-tower platform (STP) and was developed in-house. It employs a sensor array that endows the platform with a spatial-resolution capability. Given the difficulty of collecting data involving animal motion, a simulation tool was created with the aid of Blender and OpenGL software that is capable of quickly generating streams of human and animal-intrusion data. The generated data was then examined to identify a suitable collection of features that are useful in classification. The features selected corresponded to parameters that model the received signal as the superimposition of a fixed number of chirplets, an energy signature and a cross-correlation parameter. The resultant feature vector was then passed on to a Support Vector Machine (SVM) for classification. This approach to classification was validated by making use of real-world data collected by the STP which showed both STP design as well as classification technique employed to be quite effective. The average classification accuracy with both real and simulated data was in excess of 94%

Challenges in Developing and Deploying a PIR Sensor-Based Intrusion Classification System for an Outdoor Environment

The aim of the current paper is to describe the challenges faced during the development and deployment of a PIR sensor-based intrusion classification system in an outdoor environment. Some potential solutions to overcome these challenges are also presented. The challenges are of three types: (a) challenges in designing a sensor tower platform that is physically aligned to the accuracies demanded by the application, (b) challenges involved in implementing high-accuracy algorithms on a mote and, finally and perhaps most importantly, (c) challenges faced in attempting to operate a PIR-based sensor tower platform at temperatures close to that of the human body. A test deployment over a 15-week period, beginning Feb 6, 2016, was carried out within the campus of the Indian Institute of Science, Bengaluru. This period of deployment coincided with the peak summer season at which time the ambient temperature in Bengaluru ranged from 15 degrees C to 38 degrees C. This made it possible to study the behavior of the PIR-based sensor tower platform in settings where the ambient temperature is close to that of the human body